Skip to content

feat(aidd-context): add Kilo-native generation targets - #992

Open
waewoo wants to merge 42 commits into
ai-driven-dev:nextfrom
waewoo:feat/kilo-native-generation-914
Open

waewoo wants to merge 42 commits into
ai-driven-dev:nextfrom
waewoo:feat/kilo-native-generation-914

Conversation

@waewoo

@waewoo waewoo commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

🎯 What & why

Adds native Kilo support to aidd-context, enabling Kilo projects to onboard AIDD and generate discoverable project memory, rules, skills, agents, and workflows while preserving compatibility with other supported tools.

🛠️ How it works

  • Detection and memory: Recognizes six Kilo signals: .kilo/, root kilo.json / kilo.jsonc, .kilo/kilo.json / .kilo/kilo.jsonc, and legacy .kilocode/. Neither AGENTS.md alone nor OpenCode-only configuration triggers Kilo detection. Shared AGENTS.md project memory preserves user-owned content.
  • Rules: Publishes categorized .kilo/rules/<category>/<slug>.md files and registers their exact paths in Kilo instructions. Legacy .kilocode/ projects write new rules only to canonical .kilo/ paths. Unsupported path-scoped rules are rejected before any mutation rather than silently converted to non-enforceable prose.
  • Configuration and safety: Handles JSON/JSONC configuration candidates, explicit selection, ambiguous ownership, preflight checks, staged writes, and best-effort rollback. Tests cover configuration preservation, hardlinks, non-regular paths, late failures, and temporary-file cleanup.
  • Skills: Supports native .kilo/skills/ generation and explicitly selected portable .agents/skills/ placement.
  • Agents and workflows: Generates native Markdown artifacts under .kilo/agents/ and .kilo/commands/.
  • Hooks: Provides Kilo JavaScript/TypeScript plugin guidance without generating unsupported declarative hooks.
  • Cross-tool improvements: Hardens shared project-memory filesystem operations, rule publication, and hook compatibility checks.
  • Skill evaluation: Extends skill-eval with Claude/Codex support, JSON checks, project-tree snapshots, and explicit PASS / FAIL / INCOMPLETE reporting. Invalid --jobs and --repeat values are rejected before workers or evaluations start.

The rule-generation guidance keeps its detection contract within the skill, preserving standalone installation compatibility.

Implementation details and host contracts are documented in the [aidd-context README](https://github.com/ai-driven-dev/framework/blob/3c5d574545da673a61110d435bf42ec1892aab40/plugins/aidd-context/README.md).

🧪 How to verify

Run the focused Kilo CLI tests:

cd cli
pnpm test:e2e:kilo
pnpm test:e2e:kilo:native

Validation evidence:

  • All applicable GitHub Actions checks passed on PR HEAD 3c5d574; conditional release jobs were skipped.
  • Linux Kilo CLI 7.7.5 runtime tests use a deterministic local provider.
  • Authenticated Codex CLI 0.162.1 successfully generated a native Kilo skill, action, and synthetic payload using AIDD.
  • Linux Kilo 7.8.8 runtime validation confirmed skill discovery, loading, action reading, and payload consumption.
  • Targeted publication tests cover legacy detection, JSON/JSONC selection, ambiguity refusal, idempotence, hardlinks, file identity, preflight, and rollback.
  • Targeted tests also cover invalid skill-eval execution counts and standalone skill-link boundaries.

These are scoped behavioral and runtime validations, not a claim that the full skill-eval --host=codex suite passed.

⚠️ Heads-up

The AC references below correspond to the acceptance criteria defined in #914.

Maintainer decisions requested

1. PR scope — cross-tool changes

Besides native Kilo generation, this PR includes shared project-memory and rule-publishing hardening, hook compatibility improvements, and skill-eval extensions.

Should these changes remain together in #992, or should some independent improvements be extracted into follow-up PRs?

2. AC16 — Official documentation traceability

Official Kilo paths, formats, sources, and verification provenance are centralized in [official-sources.md](https://github.com/ai-driven-dev/framework/blob/3c5d574545da673a61110d435bf42ec1892aab40/aidd_docs/tasks/2026_10/2026_10_10_kilo-native-generation-914/official-sources.md).

Issue #914 records September 25, 2026, as the requested verification date; the documentation was independently rechecked on October 13, 2026.

Does this centralized evidence satisfy AC16, or is per-claim traceability required?

3. Authenticated Claude validation

CONTRIBUTING.md requests testing with Claude and another tool. Authenticated Codex generation and Kilo runtime validations are available, but no authenticated Claude Code generation session has been performed.

Can the available Codex/Kilo evidence be accepted, or could a maintainer perform the missing Claude validation?

Remaining acceptance-criteria coverage

The following acceptance criteria from #914 remain partially demonstrated. These gaps remain tracked under the original issue unless explicitly associated with a separate follow-up below.

Maintainer guidance on whether additional evidence is required before merging would be appreciated.

  • AC1–4 — Detection: The six Kilo signals, negative OpenCode/AGENTS.md cases, and legacy rule publication have targeted coverage. Complete behavioral validation across every applicable generator is not established.

  • AC7 — Configuration precedence: JSON/JSONC candidates, ownership, explicit selection, and ambiguity handling are tested. Complete Kilo runtime precedence is not established.

  • AC13 — Idempotence: Deterministic publication paths have replay tests. Idempotence of an entire LLM conversation is not guaranteed. No separate issue has been created for this broader interpretation of the acceptance criterion.

  • AC14 — Write safety: Preflight, refusal paths, hardlinks, and best-effort rollback are tested. The separate project-memory corruption defect involving I/O write failures is tracked in [#1018](fix(aidd-context): preserve project-memory files on write failure #1018). This PR does not claim to resolve that issue or guarantee crash recovery, concurrent-mutation safety, or atomic multi-file updates.

  • AC15 — Runtime coverage: Kilo runtime validation remains Linux-based. Native Kilo runtime validation on macOS and Windows is outstanding.

AC16 and authenticated Claude validation are addressed in the maintainer decisions above.

The remaining gaps do not necessarily indicate confirmed implementation defects. Some concern the scope of required guarantees or the completeness of the available evidence.

Related PR and independent follow-ups

Related integration:

Independent follow-up issues:

These issues are tracked independently. This PR does not claim to resolve them, and their existence does not automatically mean that every corresponding acceptance-criteria gap requires a separate implementation.

Maintainer review and the outstanding decisions remain necessary before merging.

🔗 Linked issue

Refs #914

✅ I certify

  • I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

Comment thread scripts/__tests__/context-kilo-detection.test.js Fixed
Comment thread scripts/__tests__/context-kilo-detection.test.js Fixed
Comment thread scripts/__tests__/context-skill-artifacts.test.js Fixed
Comment thread scripts/__tests__/update-memory.test.js Fixed
Comment thread scripts/__tests__/update-memory.test.js Fixed
Comment thread scripts/__tests__/update-memory.test.js Fixed
Comment thread scripts/__tests__/update-memory.test.js Fixed
@waewoo

waewoo commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

Quick progress update on #992:

  • Phases 1, 3, 4, and 5 are complete, with CI passing.
  • Phase 2 remains partially implemented, pending coordination with fix(aidd-context): generate rules through a standalone skill script #979 to reuse the shared rule writer.
  • Phase 6 is still pending. Authenticated Claude runtime validation is currently unavailable in my environment, so I’m not claiming that coverage.
  • The memory-sync workflow now uses a constrained manual process, without claiming the executable hook's hardlink and file-identity protections.

I've opened #994 to track the underlying plugin resource-resolution limitation separately, without expanding the scope of this PR.

I'll keep #992 in Draft until the remaining work is addressed.

Comment thread scripts/skill-eval.mjs Fixed
Comment thread scripts/__tests__/kilo-rule-publication.test.js Fixed
Comment thread scripts/__tests__/kilo-rule-publication.test.js Fixed
Comment thread scripts/skill-eval-snapshot.mjs Fixed
@waewoo

waewoo commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor Author

The implementation is ready for maintainer review. The shared rule publisher is now integrated (#979 is no longer blocking), and CI is green, including Kilo 7.7.5 Linux E2E, Windows CLI, mutation tests, and CodeQL.

I'd appreciate your feedback on three points:

  1. Cross-tool scope

Beyond Kilo-native generation, this PR includes:

  • Shared rule publishing with preflight, configuration preservation, and best-effort rollback.
  • Project-memory filesystem hardening and hook compatibility checks.
  • "skill-eval" extensions: Claude/Codex host support, JSON checks, snapshots, and "PASS"/"FAIL"/"INCOMPLETE" reporting.

The first changes support Kilo integration and existing host compatibility. The "skill-eval" extensions are less directly related.

Would you prefer to keep everything together or extract the independent changes into a follow-up PR?

  1. AC16 — Kilo documentation provenance

AC16 requires an official source and the September 25, 2026 verification date for every Kilo path claim.

I centralized the references in "official-sources.md" (https://github.com/ai-driven-dev/framework/blob/ccd32e3bd9075cce43be6bea4a7bb1c60a14e6dd/aidd_docs/tasks/2026_10/2026_10_10_kilo-native-generation-914/official-sources.md), recording the actual October 13 re-verification rather than claiming independent evidence for September 25.

Does this centralized approach satisfy AC16's intent?

  1. Claude validation

"CONTRIBUTING.md" requires testing with Claude and another tool. I don't have a Claude subscription or budget for paid API usage.

However, an authenticated Codex CLI 0.162.1 run successfully generated a native Kilo skill using AIDD, and a subsequent Kilo 7.8.8 Linux runtime test verified discovery, loading, and payload consumption. This is a scoped validation, not a full "skill-eval" suite pass.

Would you accept this Codex/Kilo evidence as an exception to the Claude requirement, or could a maintainer perform the Claude validation?

Follow-up issues

Three follow-up issues are tracked separately:

None of these issues is claimed as resolved here.

Other validation limitations are documented in the PR description.

Thanks for your guidance before merging!

@waewoo
waewoo marked this pull request as ready for review October 14, 2026 05:46
@waewoo
waewoo requested a review from a team as a code owner October 14, 2026 05:46
@blafourcade blafourcade added this to the This week milestone Oct 22, 2026 — with Claude
waewoo added 17 commits October 27, 2026 09:33
Cover six canonical and legacy signals without false Kilo detection from shared AGENTS or OpenCode config. Keep OpenCode detection unchanged.

TASK 1 of ai-driven-dev#914 phase 1. Detection contracts: 24/24 pass; removing the nested JSONC signal causes a targeted failure.
Reuse the shared AGENTS destination and preflight all selected files before Upsert and Fill. Preserve user bytes and keep automatic hooks best-effort. Reject known unsafe destinations before any selected write.

TASK 2 of ai-driven-dev#914 phase 1. 40 memory tests and 24 detection tests pass. Final make check: 604 pass, 2 existing skips, 0 fail. Real Kilo reads linked memory and applies its instruction at reported cost zero.
Record TASK 3 runtime proof, review, assertions, task statuses and resumption decisions for ai-driven-dev#914 phase 1. Preserve the approved plan and its previous snapshot; later phases remain pending and require separate authorization.

Native Codex generated shared memory. Real Kilo read the linked file and applied its instruction at reported cost zero. Final checks: 64 targeted pass; 604 full-script pass, 2 existing skips; review approve. Claude authenticated runtime and standalone Codex CLI limitations remain explicit.
Require an explicit portable choice and resolve shared target conflicts before
publication. Preserve host field contracts and user resources with a full
preflight, and record actual caller generation and Kilo runtime evidence.

Scope is limited to validated phase 3. The overall plan remains in progress;
Claude authentication, other Kilo platforms and subsequent phases remain open.

Refs ai-driven-dev#914
Implement phase 4 of ai-driven-dev#914 with Kilo contracts, regression coverage, and Linux runtime proof.
docs: archive phase 5 validation reruns

Record targeted Persona and pinned Kilo runtime evidence for ai-driven-dev#914.

test(aidd-context): prove hook contract assertions

Exercise seven targeted source-contract mutations for ai-driven-dev#914 and tighten the two assertions that survived initial probes. Archive final phase-5 validation and update the coverage matrix and review.
Verify project-file descriptors while preserving user content and metadata, and refuse detected substitutions and hard-linked targets.

Use private test fixtures and descriptor-backed witnesses. Make line-ending, permission, mode, and parent-identity expectations explicit across platforms.

Refs ai-driven-dev#914
Extend the canonical rule publication flow for Kilo while preserving project configuration and existing host behavior.

Refs ai-driven-dev#914
waewoo and others added 16 commits October 27, 2026 09:39
Document merged Kilo instructions and cover coexistence in the real 7.7.5 runtime.

Refs ai-driven-dev#914
Keep the plugin overview focused on shared behavior and avoid repeating the Kilo config-selection contract. Remove private runtime identifiers from retained task evidence.

Refs ai-driven-dev#914
Document standalone detection and official Kilo sources. Add rerun and collision checks to support ai-driven-dev#914 acceptance review.
@waewoo
waewoo force-pushed the feat/kilo-native-generation-914 branch from c0f51a1 to 5f2fcb0 Compare October 27, 2026 09:56
Comment thread scripts/__tests__/context-kilo-detection.test.js Fixed
Comment thread scripts/__tests__/context-kilo-detection.test.js Fixed
Exercise ambiguous configuration selection and predictable publication failures without claiming crash recovery or concurrent atomicity.

Refs ai-driven-dev#914
Comment thread scripts/__tests__/kilo-rule-publication.test.js Fixed
Comment thread scripts/__tests__/kilo-rule-publication.test.js Fixed
@waewoo

waewoo commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Progress update

PR #992 has been rebased onto upstream/next (829a5db6), and the latest improvements and fixes have been pushed (3c5d574).

Changes completed:

  • Path-scoped rules: The Kilo-native rule publisher now explicitly rejects unsupported paths rules before any filesystem mutation, rather than converting them into non-enforceable prose.
  • Kilo detection: Clarified the six detection signals, legacy .kilocode/ handling, and OpenCode/AGENTS.md negative cases.
  • Legacy rule publishing: Added behavioral coverage confirming that legacy Kilo projects generate rules under canonical .kilo/rules/ paths.
  • Reliability: Strengthened JSON/JSONC configuration selection, ambiguity handling, preflight checks, hardlink preservation, and rollback tests.
  • Additional fixes: Hardened skill-eval execution-count validation and corrected standalone skill documentation references.

Validation: All applicable CI checks pass on the latest PR HEAD, including the targeted Kilo validations.

The following maintainer decisions remain open:

  1. Should the cross-tool changes remain in feat(aidd-context): add Kilo-native generation targets #992 or be split into separate PRs?
  2. Does the centralized official Kilo documentation satisfy AC16 traceability requirements?
  3. Can the available authenticated Codex/Kilo validation substitute for an authenticated Claude run, or could a maintainer provide that validation?

The remaining acceptance-criteria gaps and follow-up issues are documented in the PR description.

Thanks for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants